Model Hyper Parameters

Training dates 2016-09-01 to 2019-12-30

Transformations

Channel Diminishing Returns Shape Decay rate
TV log 0.98
Digital 0.9 0.3
SEM Brand 0.2 0.4
SEM Non-Brand 0.9 0.4
Social 0.9 0.6
App 0.9 0.93

Model Performance

\(R^2\) = 0.891834

MAPE = 4.17%

MAPE (Test) = 4.34%

RMSE = 2,377

RMSE (Test) = 2,426

Estimate t value Pr(>|t|)
(Intercept) -962.624 * -0.377 0.706
tv 6.539 2.946 0.003
semBR 154.551 2.409 0.016
semNB 0.039 2.648 0.008
app 0.022 4.560 0.000
social 0.063 4.449 0.000
NA NA NA NA
email_marketing 0 * -0.727 0.467
email_operational 0.001 5.338 0.000
email_savedsearch 0 * 0.219 0.827
SEO_bigcity 2659.025 3.037 0.002
SEO_hdp 2409.648 * 1.719 0.086
push_android 0 -2.775 0.006
push_ios 0 3.089 0.002
NA.1 NA NA NA
nonbrand_queries 0.308 16.008 0.000

*Not significant effect

Lift Analysis

Lift is the expected contribution of the channel to the overall prediction. “If we removed the channel, what percent of total Submits would we lose?”
2017 2018 2019
(Intercept) -2.1% -2.2% -2.2%
tv 5.8% 7.0% 7.0%
semBR 3.4% 3.6% 3.6%
semNB 2.0% 2.4% 2.3%
app 3.6% 5.0% 7.6%
social 3.0% 3.7% 6.3%
digital NA% NA% NA%
email_marketing -0.2% -0.1% -0.1%
email_operational 5.9% 3.7% 5.4%
email_savedsearch 0.1% 0.2% 0.2%
SEO_bigcity 4.3% 5.1% 3.4%
SEO_hdp 3.6% 3.5% 3.0%
push_android -1.9% -1.3% -1.9%
push_ios 1.1% 1.9% 3.4%
active_listings NA% NA% NA%
nonbrand_queries 74.0% 75.2% 78.6%

CPA Analysis

date tv semBR semNB app social digital
2016-09-01 $36.96 $5.06 $45.57 $8.01 $16.42 Inf
2017-01-01 $41.37 $5.41 $44.68 $8.38 $17.36 Inf
2018-01-01 $38.80 $5.84 $45.32 $8.83 $17.71 Inf
2019-01-01 $32.96 $6.46 $45.21 $8.90 $18.79 Inf

Error Plot

Variable Analysis

Variable Correlation

Media Marginal Plots

The “from June” color scale is meant to be used as a visual heuristic representing peak season. The equation is absolute value of the month number minus 6. \[from\_june = Abs(month - 6)\]

Variable plots

Owned

## [1] "email_marketing"   "email_operational" "email_savedsearch"
## [4] "push_android"      "push_ios"

Apendix

Seasonal Variable

Estimate t value Pr(>|t|)
christmas -9,454 -9.768 0.000
easter -4,634 -4.474 0.000
fathers -2,619 -2.517 0.012
halloween -1,943 * -1.540 0.124
independence -9,288 -6.286 0.000
labor -2,367 -3.145 0.002
martin 1,024 * 0.712 0.477
memorial -3,532 -3.998 0.000
mothers -3,206 -3.069 0.002
new -6,421 -6.185 0.000
presidents 3,076 2.134 0.033
super -665 * -0.459 0.646
thanksgiving -4,192 -5.882 0.000
valentines -4,883 -3.408 0.001
year2017 -517 * -1.387 0.166
year2018 -2,668 -5.217 0.000
year2019 -6,796 -8.855 0.000
month2 2,355 5.135 0.000
month3 1,665 3.446 0.001
month4 1,423 2.465 0.014
month5 -1,938 -3.012 0.003
month6 -1,776 -2.720 0.007
month7 -608 * -0.916 0.360
month8 -843 * -1.612 0.107
month9 -2,669 -5.789 0.000
month10 -3,595 -8.345 0.000
month11 -4,956 -11.094 0.000
month12 -7,087 -15.731 0.000
wdayMonday 3,383 8.274 0.000
wdaySaturday -162 * -0.454 0.650
wdaySunday -1,988 -4.364 0.000
wdayThursday 1,680 4.756 0.000
wdayTuesday 2,723 7.104 0.000
wdayWednesday 2,542 6.954 0.000